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(54) Facial identification 



(57) A method of identifying a facial image records characteristic facia! features from a facial image, 
compares the recorded facial features with a database of similar characteristic facial features derived from a 
reference set of facial images, and determines from the comparison the statistical probability that any 
combination of the recorded facial features may occur in any two persons. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 

This print takes account of replacement documents submitted after the date of filing to enable the application to comply 
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'/3 




Apmarafas amid Methods Relating to Facial Identification 



This invention relates to the identification of persons by reference to characteristic fecial 
features. 

Facial recognition by an automated system or machine has in recent years become a growing 
field of research for both academics and engineers alike. The growth of this technology has 
been aided by the advent of more powerful and capable computing hardware, software 
techniques and surveillance systems which have enabled practical and usable recognition 
systems to be developed. Such systems are predominantly being developed for purposes such 
restricted entry door security and access to secure computing networks, along with research 
into areas such as fecial recognition for use in access sequences to bank ATM machines. By 
their very nature, such systems tend to operate on a restricted set of candidate feces and as 
such the recognition element of such systems is limited to that particular application. 

Due to the growing need in the criminal justice system attendant on the growth of surveillance 
imaging, a body of research in the field of recognition systems has been directed towards fecial 
identification of suspects for use injudicial proceedings. In the case of criminal prosecutions 
where the identity of a suspect is at issue, the level of certainty required for a conviction is 
defined by the British legal system as being ' beyond all reasonable doubt'. Many legal 
arguments and case specific issues may affect a jury's decision but, more recently, 
photographic and/or surveillance video information is beginning to play an increasingly 
important role in helping the courts reach decisions. Recent court rulings have suggested that 
when computer based facial image analysis systems are used for judicial purposes, a 
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quantifiable probability of the chosen system producing what is known as 'false positives* is 
required, i.e. the wrong person 'positively' identified as the criminal. It will be appreciated that 
in terms of public confidence in both the police force and the judicial system, the reduction and 
eventual elimination of false positive identifications is of great importance. 

Recognition systems have been proposed for a range of applications but to date none have 
sought to quantify the probability of a false positive identification being made when the 
method employed relies on the statistical identification of selectable comparative facial 
features. This point is the particularly important when considering the use of such techniques 
applied to pictures or video stills intended for use as court evidence. 

State of the art fecial analysis systems tend to deal with the application known as recognition. 
In this sense recognition relates to the selection of one face from a restricted group of 
candidate faces as being a match for the particular face under analysis. Recognition systems, 
such as that detailed in WO 92/08202 Al are designed to extract a global statistical 
description of an unknown face in order to enable such a match comparison to be made 
between that description and a set of similarly constructed descriptions of a restricted set of 
faces. The system works by comparing some numerical values of the statistical description or 
global descriptor of an unknown face with those descriptors of a candidate face. If the 
comparison reveals agreement within a selected threshold then the system declares the 
unknown face as recognised. 

Apparatus for extracting characteristic facial points from an image and applying a technique 
known as edge extraction to inputted facial image data is disclosed in European patent 
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application No. 0 552 770 A2. The technique centres on a method for the analysis of facial 
features related to changes in the brightness gradient of adjacent pixels analysed from a 
suitably digitally stored image. This boundary information is then compared with a limited list 
of predetermined feature shapes, which are anticipated as being located in a particular region 
of the facial image under analysis, whereupon an automatic selection of one such feature 
shapes from the list is made. No details of the 'person identification system 1 discussed in the 
introduction to EP 0 552 770 A2 is disclosed therein and furthermore no method of 
determining the probability of false recognition is given. There are of course many applications 
for such a system, but the automatic recognition of facial characteristics by such a method 
would not carry the level of certainty required for a system designed for the forensic 
comparison of facial images. 

A face classification system proposed for use in extracting a face from a two dimensional 
representation of a three dimensional scene is described in European patent application No. 0 
55 1 94 1 A2 . The system proposes a method of rotation, scaling and grey level intensity 
invariant representation of a facial image to produce a substantially unique Yeature vector 1 
therefrom. This 'feature vector 1 can subsequently be analysed for statistical correlation with a 
set of known 'feature vectors'. This type of system should be referred to as a pure recognition 
system by virtue of the fact that its purpose is to recognise and match data collected from a 
recorded facial image, in order to give a output control signal of either 'face recognised* or 
not as the case may be. The system makes no reference to measurements of statistical 
probability or accuracy in relation to any comparison made, nor does the disclosed invention 
make use of a comparative database of collected known human fecial features on which to 
base it's recognition decisions. 

3 



European patent application No. 0 582 989 discloses a system for recognising a person from a 
limited set of up to 100 people. The invention discloses two methods of characterising 
automatic recognition. The first is the iconic, based on comparisons of images with stored 
templates, and the second geometric, which is concerned with the computation coefficients to 
form 'reference vectors' for facial images to allow subsequent comparison with a set of such 
vectors for known people. The prior art system makes use of both of the above recognition 
techniques, but is intended for use in a 'yes or no 1 type recognition scenario and is furthermore 
dependant on standardised viewing conditions for the face under analysis. Applications for 
such a system may include use as a control means for a door access control system in which a 
repeatable viewing position for a face under analysis is possible. 

The present invention provides an identification system capable of analysing two facial images 
and quantifying the probability that they are the same person based on a reference to a 
database of facial features. 

It is therefore an object of the invention to provide both apparatus and a method to enable the 
quantitative comparison of facial images, based on the analysis of physical features and 
standardised facial measurements taken from said facial images, with reference to a database 
of measured human facial characteristics, such that the system provides a quantifiable estimate 
of the probability of any identification made being falsely positive. 

Accordingly there is provided a method of identifying a facial image, comprising the steps of; 
recording characteristic facial features derived from a fecial image; comparing said recorded 
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facial features with a database of similar characteristic facial features derived from a reference 
set of facial images, and determining from the comparison, the statistical probability that any 
combination of said recorded facial features may occur in any two persons. 

The invention will now be described by way of a non-limiting example, with reference to the 
following drawings, in which; 

Figure 1 shows a flow diagram of system in accordance with the invention. 

Figure 2 is a diagrammatic representation of a face with selected features indicated 

Figure 3 is a diagrammatic representation of a face with selected fecial landmarks 
indicated 

In figure 1 , a flow diagram of a system operating in accordance with the invention is shown. 
An image data capture and storage means 2, which could comprise a digital scanner or digital 
camera , is used to capture an image for forensic comparison. The image is loaded into a 
storage means such as CPU RAM to enable the data to be stored in a manner that enables 
repeatable use of such data for the purposes of possible future identification. 

The facial feature characterisation system 8 is designed for use by a trained operator, the 
interface with said operator being via a display and interaction means 4, which would typically 
take the form of a computer terminal and screen. The image for assessment is displayed on 
said screen and the operator is presented with typically three modes relating to separate 



methods of describing facial characteristics, namely facial feature selection 10, facial 
landmark selection 12 and facial angle selection 14. 

The facial feature characterisation system 8 allows the operator to make and record facial 
feature selections based matching descriptive features to the image taken from a number of 
predetermined facial features, an example of which is shown in figure 2. The fecial landmark 
selection mode allows the operator to make and record a series of prede&eranined 
measurements of particular facial characteristics examples of which are shown in figure 3 . The 
facial angle selection mode allows the operator to make and record a series of measurements 
based on the angles of projected lines oriented on specific features of the facial image under 
analysis. 

A Quality Assessment Module 16 connected to the facial feature selection means 8 is 
configured to analyse the operator selected measurements and assess the quality of the 
measurement process. The quality is quantified by using an assessment method that tests the 
operators ability to repeat any measurements taken, by requiring said operator to perform 
certain measurements more than once. This measurement of repeatability is an essential 
feature of the process due to the stringent certainty requirements required for use by the 
judicial system. Once the details of the images have been captured and stored, the images may 
then, if required, be passed to the viewpoint correction module 6. 

The picture to picture comparitor means 22 enables the operator to request the system to 
compare features selected using the various modes of the facial feature selection system 8 with 
a 'known criminal 1 database 24. This database 24 contains details of measured facial features 
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of known persons faces. The comparitor means 22 enables the system to select one or more 
faces from it's database of known faces as a possible match or matches for the face under 
analysis. 

The identification means 26, enables the operator to request the system to compare the 
features selected using the various modes of the facial feature selection system 8 with a 
•reference population facial feature database' 28. The database 28 contains the facial 
measurements, angle and feature classifications as recorded from a range of the population. 
The selected features included in the database 28 are those available to the operator to select 
when using the facial feature selection system 8 when operating on the facial image under 
analysis. 

The viewpoint correction module 6 is used to correct any angular errors present in images 
supplied for comparison, for example from surveillance cameras placed at obscure angles. 
Such errors may render captured facial images not immediately suitable for analysis, but the 
viewpoint correction module 6 helps to correct or compensate for such errors so as to enable 
the subsequent measurement of the particular facial characteristics required for the 
identification process. . 

The identification module 26 uses the results of the facial feature measurements, angles and 
classifications, made and recorded by the operator, in relation to the two faces presented for 
comparison, and calculates the statistical significance of any similarities found in the said facial 
feature measurements, angles and classifications of both images. 
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The significance of any similarities between images under analysis is quantified by the use of 
two recognised statistical methods, calculation of the percentage of the entries on the 
database 28 that match the particular features, angles or measurements selected for 
comparison, and the calculation of the probability of any pair of entries having matching 
selected facial features, angles or measurements. 

When used as an identification system in judicial proceedings the following sequence of events 
illustrates an example of how the system could be utilised. If surveillance video camera 
footage from the scene of a crime has been recorded, then the police may have a particular 
suspect whom they believe to have committed the crime. For a particular case to stand a 
chance of progressing, the police may require further identification evidence to support their 
claim. Once detained, the suspect's face will be photographed in both anterior and profile 
views, and entered onto the system using the image capture and storage means 2. 

A trained operator using the facial feature selection system 8 would recall the picture taken 
from the subject under suspicion. Using the system, the operator will then go through the 
process of characterising the facial features of the recorded picture by using the three modes 
available, namely 'facial feature selection* 10, Tacial landmark selection 1 12 and 'fecial angle 
selection 1 14. 

Figure 2 shows a typical profile view of a face 32 under analysis as presented to an operator 
using the invention. The face 32 is presented in profile during one part of the fecial feature 
selection process 10 such that features such as the nose and ears may be characterised. The 
operator selects features he can clearly define and matches these to a range of presented 
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features, examples of such selections including noses 34 and ears 36. A number of such 
features are selected in both profile and anterior view by the operator to enable a characteristic 
facial feature reference to be compiled for the face under analysis. 

Figure 3 shows a typical profile view of a face 38 under facial landmark analysis. The face 38 
is show presented in profile such, that the operator can select a facial landmark relating to the 
outer canthus of the eye 40. The operator is then requested to select the point of the outer 
canthus 40. The operator then moves to further landmark areas of the face as defined by the 
system in both profile and anterior views, which may include amongst other measurements 
nose width, mouth width and ear protrusion. 

A similar process of anterior and profile views is required for the analysis of the third facial 
characteristics selection mode, namely facial angle selection 14. This mode requires the 
operator to move and orientate and locate projected lines onto specific facial features of the 
displayed facial image under analysis. The positioning of these lines in relation to specific facial 
features forms characteristic angles for a particular face. 

During facial feature characterisation process 8, the quality assessment module 16 records 
and assesses the inputs of the trained operator. This process is conducted in order to quantify 
the quality of the responses and inputs made to the system. This process involves the 
structured repetition of certain measurements and feature selections in order to assess the 
repeatability of the operators responses. This enable the system to give a case by case 
quantitative estimate of the quality of the analysis which can be presented to the courts as 
evidence of the comparative quality of each particular identification. 
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The information recorded from all threes modes 10, 12 and 14 is fed into a reference 
population facial feature database 28 whereupon it is stored along with all other recorded 
facial reference information which has undergone the same process. Once the process of 
entering the fist image onto the system is complete, the operator may then enter the crime 
scene image, with which the suspect is to be compared. 

The identical process using of the facial feature selection system is applied to the crime scene 
image and thus the facial image data relating to each of the selection modes, 10, 12 and 14 is 
recorded. The picture taken at the crime scene may show the criminal with his face at an 
angle to the camera and as such the viewpoint estimation means 6 may be used after the facial 
feature characterisation process 8 has been completed to correct any distortions due to the 
viewpoint that may be present 

Utilising the Identification' element of the described system, once the crime scene image has 
undergone the operator controlled selection system 8 process, the system can then be 
requested to compare the data extracted from the crime scene image with that of the suspect 
whose data has previously been entered. The system then compares the characteristic features 
of both facial images and calculates the percentage match of the selected features compared to 
the reference population facial feature database 24 , for example what percentage of feces in 
the reference database 24 have the same ear shape combined with nose width combined with 
eye spacing. The use of the combination of selected features provides a very accurate 
quantitative method of assessing the likelihood of whether the person pictured at the crime 
scene is in fact the suspect who has been subsequently subjected to fecial feature analysis. 
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The system further calculates the 'pair score' for the two images under comparison, namely 
the probability, based on the information contained in the reference database 24, that the 
features selected for analysis would occur in any two persons. It is this particular statistic that 
enables the operator, and in turn the courts to assess the likelihood of false positive 
identification by giving a quantitative estimate of the chance of the suspect and criminal being 
the same person. 

Alternatively, the results of the feature classification process conducted on a surveillance 
image can be compared to a 'raid and fraud' database 24. This database 24 holds the results of 
feature characterisations of known criminals and as such can be utilised by the picture to 
picture comparitor 22 to give quantitative estimates for matches of the surveillance image to 
said 'known criminals' from the database 24. The same statistical analysis methods utilised for 
the 'identification' process are applied to this type of facial image comparison and consequently 
the results are presented with a quantitative estimate of the likelihood of a match. These 
results can then be used by the police to request that person who are identified as a possible 
match be photographed and further analysed. 
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Claims 



A method of identifying a facial image, comprising the steps of; 

recording characteristic facial features derived from a facial image; 
comparing said recorded facial features with a database of similar characteristic 
facial features derived from a reference set of facial images, and; 

determining from the comparison, the statistical probability that any combination of 
said recorded facial features may occur in any two persons. 
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